Simultaneous detection of climate change and observing biases in a network with incomplete sampling

All instrumental climate records are affected by instrumentation changes and variations in sampling over time. While much attention has been paid to the problem of detecting “change points” in time series, little has been paid to the statistical properties of climate signals one ends up with after adjusting (“homogenizing”) the data—or to the effects of the irregular sampling and serial correlation exhibited by real climate records. These issues were examined here by simulating multi-station datasets. Simple homogenization methods, which remove apparent artifacts and then calculate trends, tended to remove some of the real signal. That problem became severe when change-point times were not known a priori, leading to significant underestimation of real and/or artificial trends. A key cause is false detection of change points, even with nominally strict significance testing, due to serial correlation in the data. One conclusion is that trends in previously homogenized radiosonde datasets should be viewed with caution. Two-phase regression reduced, but did not resolve this problem. A new approach is proposed in which trends, change-points, and natural variability are estimated simultaneously. This is accomplished here for the case of incomplete data from a fixed station network by an adaptation of the “Iterative Universal Kriging” method of Sherwood (2000b), which converges to maximum-likelihood parameters by iterative imputation of missing values. With careful implementation this method’s trend estimates had low random errors and were nearly unbiased in these tests. It is argued that error-free detection of change points is neither realistic nor necessary, and that success should be measured instead by the integrity of climate signals.